There are many applications in which a user may want to segment a person or object from an image and replace them with other content. For example, the user may want to replace a person or an object (including an animal) in an image with different content, such as but not limited to another version of the same person, a different person, a different object, or a virtual character.
Examples of techniques for segmenting a person or an object from an image include a Bayesian matting and a Poisson matting, where a rough mask of the object to be segmented, called a trimap, is used. See, e.g., Y. Chuang at al., 2001, “A Bayesian Approach to Digital Matting,” CVPR 2: 264-271; J, Sun et al., 2004, “Poisson matting” ACM Trans. Graph., 23(3): 315-321. Generating a trimap may require a lot of user interaction. In Lazy snapping, a user draws strokes to specify example area of the foreground and background for the segmentation. Y. Li et al., 2004, “Lazy snapping” ACM Transactions on Graphics (TOG), 23: 303-308. In GrabCut, instead of a user-specified example area, a bounding box is used to indicate where the object to be segmented might be and the algorithm can infer the salient area within the bounding box to be the foreground. C. Rother et al., 2004, “GrabCut: Interactive foreground extraction using iterated graph cuts,” ACM Trans. Graph., 23: 309-314. In SoftScissors, a user moves a stroke along the boundary of an object to facilitate segmenting the object, and may requires a lot of user interaction. J. Wang et al., 2007, “Soft scissors: an interactive tool for real-time high quality matting,” ACM Trans. Graph, 26(3): 9. In flash Matting, two images (one captured with a flash and one without a flash) are used to segment an object. A special device for capturing, which is not realistic in many practical applications, J. Sun et al., “Flash Cut: Foreground Extraction with Flash and No-flash Image Pairs,” CVPR 2007.